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DETAILED ACTION 

It would be of great assistance to the Office if all incoming papers pertaining to a 
filed application carried the following items: 

1 . Application number (checked for accuracy, including series code and 
serial no.)- 

2. Group art unit number (copied from most recent Office communication). 

3. Filing date. 

4. Name of the examiner who prepared the most recent Office action. 

5. Title of invention. 

6. Confirmation number (See MPEP § 503). 

Response to Arguments 

1 . Applicant's arguments filed 4/21/2009 have been fully considered but they are 
not persuasive. 

Applicant's argument for Claim 1 : 

-Tanida's image reconfiguring by means of arithmetic average method cannot 
obtain a high resolution image. 

Examiner's Response - The claim language does not claim obtaining a high 
resolution image. 

-Tanida's pseudo inverse method has other difficulties in real world applications. 
Examiner's Response - The claim language does not recite limitations relating to 
real world applications. 

-Tanida's method does not obtain the same level of high resolution image data. 
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Examiner's Response - The claim language does not recite limitations relating to 
resolution levels. 

-Tanida cannot achieve setting as claimed in the amended claim 1, regarding 

initial value and iterative control means. 

Examiner's Response - The claim language does not recite limitations relating to 
the setting and resolution levels. 

-Tanida does not generate the initial image of single object image under the 

preset condition of the amended claim. 

Examiner's Response - Please see analysis of amended claim. 

-Regarding the comments on page 9 of response, examiner is unclear of the 
response in lines 1-3. 

-Tanida has a fundamental limitation to obtain a high resolution image 
Examiner's Response - The claim language does not recite limitations regarding 
obtaining a high resolution image. 

-Tanida's image update means is a linear arithmetic average method and not an 
inverse process, or back projecting. 
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Examiner's Response - Tanida does update the image using back projecting 
(Page 35, Lines 18-20). 

Applicant's argument for Claims 2-4 - the claims depend on claim 1 and by 
dependency, should be allowed. 

Examiner's Response - The rejection is maintained and therefore the grounds of 

rejection for Claims 2-4 stand. 

Applicant's argument for Claim 5 - the claim recites similar elements and the arguments 
for claim 1 apply. 

Examiner's response - See responses for claim 1 . 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 1 through 5 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Tanida et al. ("Compact image capturing system based on compound imaging and 
digital reconstruction," "Tanida") in view of Tanida et al. ("Thin observation module by 
bound optics (TOMBO): an optoelectronic image capturing system," "Tanida 2000"). 
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1) Regarding Claim 1, Tanida teaches image configuring apparatus comprising 
an object image configuring means, which reads image data of a plurality of reduced 
object images from a reduced image recording means for recording image data of a 
plurality of reduced object images obtained by photographing an object as using a 
compound-eye camera that focuses a plurality of reduced object images on a photo 
detector through micro lens array having a plurality of micro lenses arrayed therein, and 
configures a single object image based on said image data and then outputs its image 

data (Page 34 Lines 1-2: A compact image capturing system called TOMBO is used which acquires a 

plurality of images through a micro lenses array, which then outputs a single image), 

wherein said object image configuring apparatus is also configured that preset in 
the conditions including the distance between object and micro lens array (Figure 8: The 
distance between the micro lens array and object (input image) is set between 90-1 20mm), using the 
aligning distance of each micro lens of lens array and the focal length of micro lens 
array (Figure 8: The distance between the lens and the object is known. Page 36, Lines 6-8: The focal 
length of the micro lens array is 650 |jm), and based o n such presetting condition, the 
magnification ratio of reducing optical system is calculated with the known distance to 

the object (Figure 8: Using the distance to the object, and the focal length, the magnification ratio is 
calculated (M = f/f-do, where f is the focal length and do is the distance to the object (90-120mm))), and 
by obtaining the relation among one pixel of the reduced image element and its 
corresponding area of the object (Page 38, Lines 16-17: The pixels of the acquired images of the 
object is mapped onto a virtual plane). 
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an iterative control means for firstly giving said initial image data to said reduced 
image estimating means as an initial value of an image data of said single object image, 
and then repetitively conducting an estimating processing of said reduced image 
estimating means as well as an updating processing of said object image updating 
means until said difference satisfying a predetermined condition, then outputting an 
image data of said single object image at the time of said difference satisfying said 
predetermined condition as a final image data of an object image (Page 39 Lines 17-18: an 

iterative algorithm is utilized which uses an evaluation function (predetermined condition) to update the 
image data of the single object Image and produces (outputting) the Image data of the single object Image 
as final Image data, as Illustrated In Figure 10). 

Tanida fails to teach wherein said object image configuring apparatus is also 
configured that said micro lens and said light receiving elements are set without 
alignment error; previously obtain the geometric transfer function Tk describing optical 
projection from the said real object to create said reduced image element and inverse 
transfer function Tk"\ wherein said object image configuring means comprising: 

a generating means of initial object image for generating an initial image data of 
a single object image based on an image data of a plurality of said reduced object 
images captured by said micro lens array under said known condition using said inverse 
transfer function Tk"^ a reduced image estimating means for estimating an estimated 
image of each of said reduced object images from an image data of a provided single 
object image which comes from said generating means of initial object image, based on 
a geometric projection process where said transfer function Tk is used; an object image 
updating means for updating an image data of said single object image provided in said 
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reduced image estimating means by projecting a difference between said estimated 
image of each reduced object images which comes from said reduced image estimating 
means and each of said reduced object images which is captured under said known 
condition of micro lens array, using said inverse process Tk"^ of said geometric 
projection process. 

However, in the same field of endeavor, Tanida 2000 teaches wherein said 
object image configuring apparatus is also configured that said micro lens and said light 
receiving elements are set without alignment error (Pages I03i & 1032, Lines 28 & i: For the 

precise adjustment of the components (without error), a special holder is fabricated), and 

previously obtain the geometric transfer function Tk describing optical projection 
from the said real object to create said reduced image element and inverse transfer 

function Tk \{ Page 1033, Lines 18-22: a relation is described between the element on the object and 
the photo-sensitive cells (image element), assigned to the transfer matrix (function) H. A pseudoinverse 
matrix H"^ is calculated (Lines 28-30) (inverse transfer function)) 

wherein said object image configuring means comprising: a generating means of 
initial object image for generating an initial image data of a single object image based 
on an image data of a plurality of said reduced object images captured by said micro 
lens array under said known condition using said inverse transfer function Tk"^ (Page 
1034, Lines 14-17: The back projection method is applied to the signals captured by the TOMBO system 
using the matrices H and H"^ (inverse transfer function)); 

a reduced image estimating means for estimating an estimated image of each of 
said reduced object images from an image data of a provided single object image which 
comes from said generating means of initial object image, based on a geometric 
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projection process wliere said transfer function Tk is used (Page 1033, Lines 18-25: The 
system matrix H (transfer function) is used to estimate eacli of tine images captured by tlie plioto-sensitive 
cells); 

an object image updating means for updating an image data of said single object 
image provided in said reduced image estimating means by projecting a difference 
between said estimated image of each reduced object images which comes from said 
reduced image estimating means and each of said reduced object images which is 
captured under said l<nown condition of micro lens array, using said inverse process Tk'^ 

of said geometric projection process (Page 1034, Lines 8-13: The back projection method 
retrieves an updated image using an inverse process). 

Applying the teachings of Tanida 2000 to Tanida's apparatus allows for using a 
transfer function and an inverse transfer function which allows for using back projection 
methods to reconstruct an image acquired by a micro lens array yielding a high- 
resolution image, and therefore a better result due to the increase in quality of the 
image. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to apply the teachings of Tanida 2000 to Tanida. 

2) Regarding Claim 2, Tanida teaches in combination with Tanida 2000 the 
limitations of claim 1 , and further teaches object image configuring means further 
comprises a shift amount calculating means for calculating a shift amount in regard to a 
gap of relative positions between said reduced object images through correlation 
calculation between said reduced object images by using an image data of a plurality of 
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said reduced object images (Page 39 Lines 2-3: each unit image (reduced images) is correlated 
with a reference image to determine the lateral offset (shift amount)). 

3) Regarding Claim 3, Tanida teaches in combination with Tanida 2000 the 
limitations of claim 1 , and further teaches object image configuring means further 
comprises a projection process deriving means for obtaining a conversion equation 
indicating said geometric projection process employed in said reduced image estimating 
means based on said shift amount obtained in said shift amount calculating means 

(Page 39 Lines 4-6: R(x, y) is a conversion equation, which is normalized correlation using f(x, y) and g(x, 
y) of the reference and unit images and the shift calculated of the shift amount calculating means). 

4) Regarding Claim 4, Tanida teaches in combination with Tanida 2000 the 
limitations of claim 1 , and further teaches generating means of initial object image in 
said object image configuring means generates an image data of a single object image 
by arranging a plurality of said reduced object images onto a same area based on said 
shift amount obtained in said shift amount calculating means, and then generates an 
initial image data of said single object image by interpolating blank pixels with respect to 

said image data (Page 39 Lines 9-10: The offset values (shift amount) of the unit images are 
calculated using the calculating means. These pixels are then mapped on the virtual image plane to 
generate image data of a single object image. Page 38 Lines 9-10: Interpolation is used to compensate 
information of the pixels which are not assigned the pixels of the unit images (blank pixels) generating an 

initial image data). 

5) Regarding Claim 5, Tanida teaches image configuring method for configuring 
a single object image based on an image data of a plurality of reduced object images 
obtained by photographing an object as using a compound-eye camera that focuses a 
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plurality of reduced object images on a photo detector through micro lens array having a 
plurality of micro lenses arrayed therein (Page 34 Lines 4-5:A new algorithm for image 
reconstruction has been developed using a TOMBO system is used which generates a plurality of 

reduced object images on a micro lens array (Figure 1)), comprising the Steps Of: 

estimating an estimated image of each of said reduced object Images from an 
image data of a provided single object image based on a geometric projection process 

using a transfer function Tk (Page 38 Lines 5-6: A pixel rearrange method is used where pixels in 
the unit images are rearranged onto a virtual image plane, which is a parallel projection, a form of a 
geometric projection process. See Figure 7 for illustration.); 

updating an image data of said single object image provided in said reduced 
image estimating process by projecting a difference between estimated images of each 

of said reduced object images and each of said reduced object images in an inverse 

process (Tk"^) of said geometric projection process (Page 39 Lines 13-14: Linear interpolation 
between valid pixels (image data) is done to update the image data. Figure 9(c) shows the geometric 
process and 9(d) shows the updated image which uses interpolation (inverse of geometric process)); 

providing said initial image data to said reduced image estimating process as an 
initial value of an image data of said single object image; repetitively conducting said 
reduced image estimating process as well as said object image updating process until 
said difference satisfying a predetermined condition; and outputting an image data of 
said single object image at the time of said difference satisfying said predetermined 

condition as a final image data of an object image (Page 39 Lines 17-18: an iterative algorithm 
is utilized which uses an evaluation function (predetermined condition) to update the image data of the 
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single object image and produces (outputting) the image data of the single object image as final image 
data, as illustrated in Figure 10). 

Tanlda fails to teach generating an initial image data of a single object image 
based on an image data of a plurality of said reduced object images using known 
conditions and an inverse transfer function Tk\ 

Tanida 2000 teaches generating an initial image data of a single object image 
based on an image data of a plurality of said reduced object images using known 
conditions and an inverse transfer function Tk'^ (Page 1034, Lines 14-17: The back projection 
method is applied to the signals captured by the TOMBO system using the matrices H and H"^ (inverse 
transfer function)). 

Generating an image using an inverse transfer function allows for using back 
projection methods to reconstruct an image acquired by a micro lens array yielding a 
high-resolution image, and therefore a better result due to the Increase in quality of the 
image. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to apply the teachings of Tanida 2000 to Tanida. 

Conclusion 

4. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 
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A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Nirav G. Patel whose telephone number is (571 )270- 
5812. The examiner can normally be reached on Monday - Friday 8 am - 5 pm EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached on 571-272-7453. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Nirav G. Patel/ 
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/Wenpeng Chen/ 
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